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Sir: 

This is an appeal from the final rejection of claims 1- 3 and 6 - 8 of 
the application as set forth in the Office Action, made final, mailed September 24, 
2003. 

REAL PARTY IN INTEREST 

The real party in interest in this appeal is Polaroid Corporation, a 

corporation organized and existing under the laws of the State of Delaware, of 1265 
Main Street, Waltham, MA 0245 1 . 

RELATED APPEALS AND INTERFERENCES 

There are no related appeals and interferences. 

STATUS OF CLAIMS 

Claims 1-3 and 6-8 have been rejected as being unpatentable over 

the references applied in support of the rejections. 

Claims 4 and 9 have been indicated as containing allowable subject 
matter but are objected to as being dependent upon a rejected claim (claims 1 and 6, 
respectively). 

Claims 5 and 10 have not been rejected or objected to. However, 
these claims are dependent upon rejected claims 1 and 6, respectively. It would 
appear in view of the subject matter of claims 5 and 10 and their similarity to the 
subject matter of claims 4 and 9 that these claims are considered to contain 
allowable subject matter but are objected to as being dependent upon a rejected 
claim. 
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STATUS OF AMENDMENTS 

No amendments to the claims have been made by applicants. 

A Response to Final Rejection was mailed March 15, 2004. The 
Advisory Action stated that the rejection of claims 3-5 and 8—10 under the first 
paragraph of 35 USC § 1 12 had been overcome. 

The Advisory Action also indicated that claims 4 and 9 contain 
allowable subject matter but are objected to as being dependent upon a rejected 
claim (claims 1 and 6, respectively). 

SUMMARY OF INVENTION 

Applellants' claimed method and apparatus are directed to recovering 
missing color data in a two-dimensional color array. The method and apparatus 
involve the application of two one-dimensional non-linear interpolation processes. 
Generally speaking, applicants' method essentially "decouples" the conventional 
two-dimensional process into two one-dimensional processes. The first one- 
dimensional color recovery process generates intermediate second color image data 
from the sampled first color image data and the second one-dimensional color 
recovery process generates the desired third color image data from the second color 
image data. 

As described in detail in the application (see, for example, page 8, 
line 28 to page 10, line 14) a two-dimensional array of discrete image sensing 
elements, each of which is specifically responsive to one of at least three 
predetermined colors (e.g., red, green and blue) is exposed to image information- 
bearing illumination to obtain a collection of each electronic information signal 
received from each discrete element. The collection of signals forms the raw 
unprocessed one-color image data from which fully recovered third color image data 
can be derived. 

The first step towards deriving the fully-recovered third color image 
data is to first recover missing color information along a first dimension (e.g., along 
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rows of the array) by interpolating the first color image data along the first 
dimension to provide a first-interpolated color data for each of the discrete 
elements, forming a difference channel between the first color image data and the 
first interpolated color data and then applying a one-dimensional non-linear filter on 
the difference channel and combining with the first color data to obtain the first 
recovered image data. The second color image data comprises a combination of the 
first recovered image data and the first color image data. 

The second step of the method derives fully-recovered third color 
image data from the two color image data by recovering missing color information 
along a second dimension, e.g., along columns of the array. The second color image 
data is obtained by interpolating along the second dimension to provide second 
interpolated data for each of the discrete elements, forming a difference channel 
between the second color image data and the second interpolated data and then 
applying a one dimensional non-linear filter on the difference channel and 
combining with the second color data to obtain the second recovered image data. 
The third color image data comprises a combination of the second recovered image 
data and the second image data. 

REFERENCES APPLIED BY EXAMINER 

1 . U.S. Patent No. 6,240,2 1 7 B 1 ("Ercan et al.") 

2. U.S. Patent No. 6,396,539 Bl ("Heller et al.") 

3. United States Patent Application Publication No. 
US2002/0081019 Al ("Katayama et al.") 

DISCUSSION OF THE REFERENCES 

Ercan et al relates to an imaging method and is directed exclusively 
to improving the contrast of a digital image to accentuate important image details. 
Areas of the image requiring improvement are identified as those with a narrow 
range of pixel values and having a median value that is significantly different than 
the median pixel value of other parts of the image. Proposed corrections act toward 
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bringing the median value of the area more toward the average value and increasing 
its contrast. In another aspect of the invention, two different cameras are used to 
take pictures at different exposures and image processing is used to provide a 
composite picture with well-exposed detail in all areas of the image. 

The image obtained by Ercan et al is not described as having any 
missing data and the processing described is for the purpose of improving the 
visibility of existing image detail whereas the method of applicants involves the 
recovery of missing data. 

Heller et al deals with correcting defective pixels on image sensing 
elements. Generally, the method of Heller et al involves using certain pixels of the 
image sensor to obtain the missing information and stores the location of the 
defective pixel and of the corrective pixels on the integrated image sensing device 
itself. Two embodiments are described (see, for example, column 8, lines 57 - 65). 
In one embodiment, compensation pixel values are generated by examining the pixel 
values surrounding the defective pixel. In the other embodiment the pixel value of a 
pixel that precedes the defective pixel is used as the compensation pixel value. 

The disclosure of the reference is silent as to the color of the pixels 
which are used to provide the compensation values; however it can be inferred that 
pixels of the same color are used to obtain the compensation values. Both 
embodiments of Heller et al use linear filters. 

Katayama et al is directed to image sensing apparatus capable of 
placing an object, whose three dimensional shape is to be generated, under the 
optimum image sensing conditions upon sensing the object from a plurality of image 
sensing points. 

ISSUES 

1 . Whether the subject matter of claims 1, 3, 6 and 8 is obvious 
under 35 USC § 103(a) over Ercan et al. in view of Heller et al. 



-5- 



Appl. No. 09/833,934 

Appeal Brief dated July 22, 2004 



2. Whether the subject matter of claims 2 and 7 is obvious under 
35 USC § 103(a) over Ercan et al. in view of Katayama et al. 

GROUPING OF CLAIMS 

Pursuant to 37 CFR 1.192(c)(7), appellants request separate 
consideration of the following groups of claims: 

(1) Claims 1,3,6 and 8 

(2) Claims 2 and 7 

The examiner has applied a different ground of rejection to each 
group and each ground of rejection should be considered separately. 

ARGUMENT 

Summary 

The references, viewed individually or in combination, do not 
disclose or suggest within the meaning of 35 USC § 103(a) a method for 
electronically capturing and processing image information which involves the 
application of two one-dimensional non-linear interpolation processes. 

Detailed Argument 

A. The rejection of claims 1, 3, 6 and 8 under 35 USC § 103(a) 

as being unpatentable over Ercan et al in view of Heller et al is not supported by the 
disclosure of the references. 

Appellants' claimed method and apparatus are directed to recovering 
missing color data in a two-dimensional color array. The method and apparatus 
involve the application of two one-dimensional non-linear interpolation processes. 
Generally speaking, applicants' method essentially "decouples" the conventional 
two-dimensional process into two one-dimensional processes. The first one- 
dimensional color recovery process generates intermediate second color image data 
from the sampled first color image data and the second one-dimensional color 
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recovery process generates the desired third color image data from the second color 
image data. 

As described in detail in the application (see, for example, page 8, 
line 28 to page 10, line 14) a two-dimensional array of discrete image sensing 
elements, each of which is specifically responsive to one of at least three 
predetermined colors (e.g., red, green and blue) is exposed to image information- 
bearing illumination to obtain a collection of each electronic information signal 
received from each discrete element. The collection of signals forms the raw 
unprocessed one-color image data from which fully recovered third color image data 
can be derived. 

The first step towards deriving the fully-recovered third color image 
data is to first recover missing color information along a first dimension (e.g., along 
rows of the array) by interpolating the first color image data along the first 
dimension to provide a first-interpolated color data for each of the discrete 
elements, forming a difference channel between the first color image data and the 
first interpolated color data and then applying a one-dimensional non-linear filter on 
the difference channel and combining with the first color data to obtain the first 
recovered image data. The second color image data comprises a combination of the 
first recovered image data and the first color image data. 

The second step of the method derives fully-recovered third color 
image data from the two color image data by recovering missing color information 
along a second dimension, e.g., along columns of the array. The second color image 
data is obtained by interpolating along the second dimension to provide second 
interpolated data for each of the discrete elements, forming a difference channel 
between the second color image data and the second interpolated data and then 
applying a one dimensional non-linear filter on the difference channel and 
combining with the second color data to obtain the second recovered image data. 
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The third color image data comprises a combination of the second recovered image 
data and the second image data. 

Ercan et al teaches steps (a) and (b) of appellants' method, which step 
are generally known in the art. However, as has been acknowledged by the 
examiner during prosecution, this reference does not teach or suggest appellants' 
claimed method. 

Ercan et al is directed to an imaging method for improving the 
visibility and legibility of a digital image. Areas of the image requiring 
improvement are identified as those with a narrow range of pixel values and having 
a median value that is significantly different than the median pixel value of other 
parts of the image. Proposed corrections act toward bringing the median value of 
the area more toward the average value and increasing its contracts. 

In another aspect of the method of Ercan et al, two different cameras 
are used to take pictures at two different exposures and image processing is used to 
provide a composite picture with well-exposed detail in all areas of the image. 

The image obtained by the method of Ercan et al is not described as 
having any missing data and the processing described is for the purpose of 
improving the visibility of existing image detail whereas the method of appellants 
involves the recovery of missing data. 

Ercan et al relates exclusively to improving visibility or legibility of a 
digital image. Fig. 6 A illustrates an enhanced 4-bit image of the license plate of Fig. 
3 as processed according to Example 1 ; Fig. 9A illustrates the same license plate as 
processed in Example 2; and Fig. 10A illustrates the same license plate as processed 
in Example 3. In none of these instances is missing information recovered. 

The examiner has stated that Ercan et al. in Fig. 5, illustrates a two- 
dimensional color array. This is not correct . The reference, at column 1 1 , lines 9 - 
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10 states "Fig. 5 illustrates the mapping arrangement used in the above [example 1] 
example". 

The examiner appears to have argued that example 5 of Ercan et al. 
illustrates a mapping arrangement wherein comparison of two images is carried out 
until the desired color image is obtained and that this process is the same as the 
method of applicants. Example 5 of this reference does not recover missing color 
data. 

Heller et al, the secondary reference, does not render the rejection any 
more effective. Heller et al deals with correcting defective pixels on image sensing 
elements. Generally, the method of Heller et al involves using certain pixels of the 
image sensor to obtain the missing information and stores the location of the 
defective pixel and of the corrective pixels on the integrated image sensing device 
itself. Two embodiments are described (see, for example, column 8, lines 57 - 65). 
In one embodiment, compensation pixel values are generated by examining the pixel 
values surrounding the defective pixel. In the other embodiment the pixel value of a 
pixel that precedes the defective pixel is used as the compensation pixel value. 

The disclosure of the reference is silent as to the color of the pixels 
which are used to provide the compensation values; however it can be inferred that 
pixels of the same color are used to obtain the compensation values. It should also 
be recognized that both embodiments of Heller et al use linear filters and there is no 
explicit teaching of the use of non-linear filters as are used in the method of 
appellants. Appellants 5 method involves the use of second color data and non-linear 
filters and the method of Heller et al does not. Appellants' method also assumes that 
data is available at all pixels and therefore that defective pixels, such as are 
addressed by Heller et al, do not exist or have been previously compensated. 

The examiner has referred to the fact that Ercan et al refers, in the 
Abstract, to using a non-linear algorithm. This teaching is in the context of the 
overall method taught by the reference. As pointed out in detail, Ercan et al does not 
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teach recovering missing color information and the examiner has acknowledged this 
fact. 

There is no suggestion in Ercan et al which would provide any 
incentive for one skilled in the art to take the reference to a non-linear algorithm out 
of the context of the overall teaching and incorporate a non-linear algorithm into the 
method of Heller et al. It is impermissible to take from one reference only so much 
as is required to combine with another reference to arrive at the claimed invention. 
There must be some suggestion in the reference to do so. Here, since the respective 
methods of Ercan et al and Heller et al are directed to different results, there would 
be no suggestion to use a non-linear filter in the method of Heller et al. In fact, if a 
non-linear filter were to be used in the method of Heller et al the intended method 
would not be operative for its intended purpose. 

In any event, even if, assuming arguendo, Ercan et al and Heller et al 
were to be combined in the manner suggested by the examiner, the combined 
disclosures do not amount to applicants' method. As has been discussed in detail, 
neither of the references involves the application of two one-dimensional non-linear 
interpolation processes. 

The examiner has stated that it is the examiner's interpretation that 
".. .correcting defective pixels on image sensing elements would have the same 
concept as correcting missing information pixels on image elements." It is submitted 
that this statement does not relate to the teaching of Heller et al. 

The issue to be addressed is whether the rejection of the claims is 
proper based on the teaching(s) of the reference(s). Here it has been shown that 
Heller et al does not teach or suggest recovering missing color information. 

From the foregoing, it is evident that the disclosures of these 
references do not place appellants' claimed method and apparatus in the possession 
of the general public as is necessary in order to properly support a rejection under 35 
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USC § 103. Claims 1 5 3, 6 and 8 are clearly patentably distinguishable over these 
references. 

B. The rejection of claims 2 and 7 under 35 USC § 103(a) as 
being unpatentable over Ercan et al in view of Katayawa et al is not supported by the 
disclosures of the references. 

Method claim 2 is dependent on claim 1 and apparatus claim 7 is 
dependent on claim 6. These dependent claims are drawn to a preferred 
embodiment of applicants' invention wherein the discrete sensing elements are 
arranged in a pattern such that (a) no two discrete elements that are contiguous along 
the first or second dimension are specifically responsive to the same one of the at 
least three predetermined colors and (b) no more than one discrete element is 
contiguously between two discrete elements that are specifically responsive to the 
same one of the at least three predetermined colors. 

In support of the rejection the examiner has generally asserted that 
although Ercan et al does not specify pattern-wise arrangement of image sensors, 
Katayama et al does teach providing an image sensing apparatus which determines 
whether the optimum image sensing conditions are achieved or not by judging 
whether there is a predetermined pattern in an image sensing field. 

Katayama et al discloses an image sensing apparatus capable of 
placing an object, whose three dimensional shape is to be generated, under the 
optimum image sensing conditions upon sensing the object from a plurality of image 
sensing points. 

The examiner has stated that "Katayama on page 3 paragraph 0082, 
in a second embodiment teach correcting missing color on the image, and also see 
Fig. 31." 

Paragraph 0082 of this reference states, with respect to a second 
embodiment that . .it is characterized by correcting a three-dimensional image on 
the basis of predetermined image sensing parameters or editing a three-dimensional 
image.". Nothing in this statement or the illustration shown in Fig. 3 1 deals with 
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appellants' claimed method and apparatus directed to recovering missing color data 
in a two-dimensional color array involving the application of two one-dimensional 
non-linear interpolation processes. Katayama et al is not concerned with recovering 
missing color information of an image. 

The deficiencies of Ercan et al with respect to Appellants' claimed 
method and apparatus have been discussed at length above. 

For all the foregoing reasons, it has been shown that the references 
relied upon to support the rejections do not do so. The references, viewed 
individually or in combination, do not teach or suggest the subject matter of claims 2 
and 7. 



For all the foregoing reasons, the rejectionsof claims 1-3 and 6 — 8 



under 35 USC 103(a) should be reversed and all the claims in the application 



Polaroid Corporation 
Patent Department 
1265 Main Street 
Waltham, MA 02451 
Tel.: 781-386-6405 
Fax: 781-386-6435 



CONCLUSION 



allowed. 



Respectfully submitted, 




Gaetano D. Maccarone 
Registration No. 25,173 
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APPENDIX 

Claims On Appeal 

1. A method for electronically capturing and processing image 
information comprising the steps of: 

(a) providing a two-dimensional array of discrete image sensing 
elements, each discrete element capable of providing an electronic information 
signal in response to incident illumination, said electronic information signal 
corresponding to the intensity of said incident illumination, each discrete element 
being specifically responsive to one of at least three predetermined colors; 

(b) obtaining first color image data by exposing the two-dimensional 
array to image-information bearing illumination such that each discrete element 
provides said electronic information signal, said first color image data comprising 
the collection of said electronic information signals; 

(c) recovering missing color information along a first dimension by (i) 
interpolating the first color image data along said first dimension to provide first- 
interpolated color data, (ii) forming a first difference channel between said first color 
image data and said first-interpolated color data, (iii) applying a first one- 
dimensional non-linear filter to said first difference channel, whereby the first- 
recovered image data is obtained as a combination of the first color image data and 
the filtered first difference channel, and iv) forming second color data comprising 
the first color data and the first-recovered color data; and 

(d) recovering missing color information along a second dimension by (i) 
interpolating the second color image data along said second dimension to provide 
second interpolated color data, (ii) forming a second difference channel between 
said second color image data and said second interpolated color data, (iii) applying a 
second one-dimensional non-linear filter to said second difference channel, whereby 
the second-recovered color data is obtained as a combination of the second color 
data and the filtered second difference channel, and iv) forming final recovered 
image data comprising the second color data and the second recovered color data. 
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2. The method of claim 1, wherein the discrete elements are 
pattern-wise arranged such that (a) no two discrete elements that are contiguous 
along said first or second dimension are specifically responsive to the same one of 
said at least three predetermined colors, and (b) no more than one discrete element is 
contiguously between two discrete elements that are specifically responsive to the 
same one of said at least three predetermined colors. 

3. The method of claim 2, wherein said first and second one- 
dimensional non-linear filters are rank-order filters. 

6. An electronic imaging apparatus comprising: 
a two-dimensional array of discrete image sensing elements for generating 
first color image data, each discrete element capable of providing an electronic 
information signal in response to incident illumination, said electronic information 
signal corresponding to the intensity of said incident illumination, each discrete 
element being specifically responsive to one of at least three predetermined colors; 

a first color recovery module for generating a second color image data from 
said first color image data, the first color recovery module having first means for 
interpolating said first color data along a first dimension to provide first-interpolated 
color data, first means for non-linear filtering and combining said first-interpolated 
color data with said first color image data in said first dimension to provide first- 
recovered color data, and forming second color data comprising said first color data 
and said first-recovered data; and 

a second color recovery for generating a final color-recovered image data 
from said second color image data, the second color recovery module having second 
means for interpolating said second color data along a second dimension to provide 
second interpolated color data, second means for non-linear filtering and combining 
said second interpolated color data with said second color image data in said second 
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dimension to provide a second-recovered color data, and forming a final recovered 
image, comprising said second color data and said second-recovered data. 

7. The electronic imaging apparatus of claim 6, wherein the 
discrete elements are pattern-wise arranged such that (a) no two discrete elements 
that are contiguous along said first or second dimension are specifically responsive 
to the same one of said at least three predetermined colors, and (b) no more than one 
discrete element is contiguously between two discrete elements that are specifically 
responsive to the same one of said at least three predetermined colors. 

8. The electronic imaging apparatus of claim 7, wherein said 
first and second means for non-linear filtering both include rank-order filters. 
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Claims Objected To 



4. The method of claim 3, wherein each discrete element is 
responsive to one of three predetermined colors, the three predetermined colors 
being a color substantially within the red wavelengths, a color substantially within 
the green wavelengths, and a color substantially within the blue wavelengths. 

9. The electronic imaging apparatus of claim 8, wherein each 
discrete element is responsive to one of three predetermined colors, the three 
predetermined colors being a color substantially within the red wavelengths, a color 
substantially within the green wavelengths, and a color substantially within the blue 
wavelengths. 

Claims Presumed To Be Objected To 

5. The method of claim 3, wherein each discrete element is 
responsive to one of three predetermined colors, the three predetermined colors 
being a color substantially within a combination the red and green wavelengths, a 
color substantially within a combination of the green and blue wavelengths, and a 
color substantially within a combination of the red and blue wavelengths. 

10. The method of claim 8, wherein each discrete element is 
responsive to one of three predetermined colors, the three predetermined colors 
being a color substantially within a combination of the red and green wavelengths, a 
color substantially within a combination of the green and blue wavelengths, and a 
color substantially within a combination of the red and blue wavelengths. 
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